Phase-based adaptive dynamic load balancing for parallel tree computation

Dynamic load balancing (DLB) is a technique for the parallel implementation of problems which generate unpredictable workloads by migrating work units to lightly loaded processors based on run-time workload measurement. Adaptive DLB is a re nement where aspects of the load balancing system itself are modi ed in the light of measured workloads. This thesis investigates phase-based adaptive DLB, a version of adaptive DLB in which a parallel computation moves through di erent load balancing phases identi ed on the basis of run-time workloads. The idea is explored through a case study of parallel tree computation, in which three distinct phases with intervening transitions are identi ed. Two major variants of phase-based adaptivity are distinguished. In parametric adaptivity, parameters of the DLB algorithm are adapted between phases; in algorithmic adaptivity, di erent DLB algorithms are utilised in each phase. These concepts are investigated quantitatively through a simulator for parametric adaptivity and discussed in detail for algorithmic adaptivity. The simulator permits a range of processor topologies, parameterises the performance of the underlying network, includes two di erent network performance models, and allows a wide range of simulated tree-structured workloads, parameterised by depth, fan-out, node granularity and imbalance. It was extensively validated in relation to the performance of two DLB algorithms on a 512-processor Cray T3D. The simulator was used to evaluate the bene t of parametric phase-based adaptivity. Preliminary experiments with non-adaptive algorithms revealed performance to be sensitive to the interval between load-balancing invocations, so this parameter was prioritised for subsequent adaptivity experiments. A performance metric called Improvement Through Adaptivity (ITA) was discussed. Two DLB algorithms were used as test cases; the well-established Generalised Dimension Exchange Method and a novel Loadserver algorithm, whose implementation is described in the thesis. Results were obtained for all combination of the transitions, and the relationships between ITA and various parameters (processor sizes, node granularity, tree imbalance and network performance) were established. Similar relationships were observed for both algorithms, though with some di erences in detail. Positive values of ITA were obtained with both algorithms, for at least one transition combination, over a range of all the parameters. Thus, the potential bene ts of phase-based parametric adaptivity are con rmed, justifying future work in implementing this technique. To my parents, Rokiah Abdul Rashid and Haron Hamid, for their unconditional love and care throughout my life. Acknowledgments I would like to deeply thank all of those who helped me with this thesis. My rst supervisor, Dr John R Davy for his invaluable advice, guidance and encouragements during the course of this work, especially during the last nine months. My second supervisor, Prof Peter M Dew, also o ered valuable comments at the early stage of this work. The Parallel Group members, Dr Jonathan Nash, Dr Wissal Essah and Dr Don Goodeve have also provided help in various ways. Their help and support are gratefully acknowledged. Thanks are also due to my postgraduate friends for their help and assistance throughout my study. I would also like to thank Mike Parsons, a nal year project student, who assisted the initial implementation of the load balancing algorithms. I am grateful to my friends and housemates for their friendship, help and encouragement. My special friend, in particular, who has always been a great support and encouragement during the last stage of my study. Finally, my special thanks and appreciations go to my parents, for their constant support and advice, encouragement and prayers, to whom this thesis is dedicated. Thanks are also due to my brother and sisters, brother-in-law, my niece and nephews who have always been wonderful to me. I gratefully acknowledge the nancial support of the Universiti Sains Malaysia throughout my research.